# Sparse Attention

A compact version of LEGAL-BERT, employing Local+Sparse+Global attention mechanism (LSG) for efficient long-sequence processing

A fine-tuned model based on bigbird-roberta-base, optimized for trivia QA tasks with long sequence processing support.

BigBird is a Transformer model based on sparse attention, capable of processing sequences up to 4096 in length, suitable for tasks like long document summarization.

A Transformer model based on sparse attention, fine-tuned specifically for mobile app description classification tasks

BigBird is a Transformer model based on sparse attention, capable of handling longer sequences, suitable for tasks like long document summarization.

BigBirdPegasus is a Transformer model based on sparse attention, capable of handling longer sequences, especially suitable for long document summarization tasks.

BigBird is a Transformer model based on sparse attention, capable of processing sequences up to 4096 tokens long, suitable for long document tasks.